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Does language influence recognition for spatial scenes? In Experiments | and 2, participants viewed ambigu- 
ous pictures, with or without spatial sentences. In a yes—no recognition task, only the spatial sentences group 
made more false alarms toward the center of the spatial category than in the other direction; three other compari- 
son groups showed no such tendency. This shift toward the core of the semantic category suggests that spatial 
language interacted with perceptual information during encoding. In Experiment 3, we varied the materials to 
test the interactive encoding account against a separate encoding account in which separately stored sentences 
are accessed during picture recognition. The results support the interactive encoding account in which spatial 
language influences the encoding and memory of spatial relations. 


In recent times, there has been a resurgence of inter- 
est in the question of whether and how language influ- 
ences thought (Bowerman & Levinson, 2001; Gentner 
& Goldin-Meadow, 2003; Gumperz & Levinson, 1996). 
Some of this work is in the classic tradition of the strong 
version of the language-and-thought hypothesis, /inguistic 
determinism: the thesis that the language a person speaks 
determines the way in which one perceives and encodes 
the world. This version is expressed in Whorf’s (1956) 
quote of Sapir: “[w]e see and hear and otherwise experi- 
ence very largely as we do because the language habits of 
our community predispose certain choices of interpreta- 
tion” (p. 134; Sapir, 1929, p. 210). For example, in one 
influential line of research, Levinson and his colleagues 
(Levinson, 1996, 2003; Levinson, Kita, Haun, & Rasch, 
2002; Pederson et al., 1998) have suggested that the spa- 
tial frames of reference employed by a given language in- 
fluence its speakers’ performance on nonlinguistic tasks. 
They report that speakers of languages that predominantly 
use egocentric spatial terms (such as /eft—-right in English) 
attend greatly to the left-right order of objects with respect 
to the speaker. In contrast, speakers of languages (e.g., 
Tzeltal) that predominantly use absolute spatial terms (the 
equivalent of north-south) encode positions with respect 
to a global framework. Levinson et al.’s (2002) conclusion 
that performance in simple nonlinguistic tasks reflects the 
dominant spatial frame in a participant’s language has gen- 
erated considerable interest as well as some challenges. 
Li and Gleitman (2002) argued that an array of strategies 
can also be induced in monolingual English speakers by 


changing the task context, to which Levinson et al. (2002) 
countered that within-language context effects are not 
inconsistent with substantial crosslinguistic differences 
in default patterns of usage (see also Majid, Bowerman, 
Kita, Haun, & Levinson, 2004). Such within-language ef- 
fects have been reported by Taylor and Tversky (1996), 
who found that changes in aspects of the scene to be de- 
scribed resulted in changes in English speakers’ choice of 
linguistic frame of reference (perspective, in Taylor and 
Tversky’s terms). Other investigations of the strong Whor- 
fian hypothesis have likewise generated a mix of positive 
findings (e.g., Boroditsky, 2001; Lucy, 1992) and nega- 
tive findings (e.g., Gennari, Sloman, Malt, & Fitch, 2002; 
Malt, Sloman, & Gennari, 2003; Papafragou, Massey, & 
Gleitman, 2002). 

The debate over strong versions of the Whorfian hy- 
pothesis, as exemplified above, may require considerable 
research before it can be settled. However, current discus- 
sions have also refined and differentiated the language- 
and-thought question (Bowerman & Levinson, 2001; 
Gentner & Goldin-Meadow, 2003). One moderate ver- 
sion is that habitual ways of talking and the construals 
habitually made for linguistic purposes become especially 
available in nonlinguistic contexts (Hunt & Agnoli, 1991). 
A still weaker version is Slobin’s (1996, 2003) thinking- 
for-speaking hypothesis, which states that linguistic influ- 
ences occur when language is used during a task (see also 
Pinker, 1989). The idea is that, in speaking, we are induced 
by the language we use to attend to certain aspects of the 
world while disregarding or de-emphasizing others. 
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Our interest here was in Slobin’s thinking-for-speaking 
hypothesis. This view may seem unremarkable, but the 
fact that we spend so much of our time using language 
(both when communicating with others and when think- 
ing within ourselves) suggests that such an influence may 
in fact be quite pervasive. In addition, this version of the 
hypothesis has the attractive feature that it permits testing 
for an effect of language on thought within a language, al- 
lowing researchers to factor out effects of culture that can 
muddy the interpretation of crosslinguistic studies. 

In a number of studies, effects of language on cognitive 
measures when language was used during the task have 
been reported. For example, Billman and Krych (1998) 
found that hearing path verbs (e.g., enter) or manner verbs 
(e.g., walk) while watching videotaped motion events influ- 
enced English-speaking subjects’ subsequent recognition 
of variants of the events: They were more likely to notice 
a change in manner if they had heard a manner verb, and 
likewise for path. Gentner and Loftus (1979) found that 
performance on a picture recognition task was influenced 
by having matched the picture to a verb 1 week prior to the 
recognition test. Participants who had matched a specific 
verb (e.g., hiking) to a general picture (a woman walking) 
were highly likely to choose the specific picture (a woman 
hiking) over the general picture they had actually received. 
The results of these studies suggest that attending to lan- 
guage during the encoding of scenes can influence scene 
encoding and subsequent recognition. Furthermore, these 
effects are not limited to motion verbs. Loewenstein and 
Gentner (2001, 2005) found that preschool children per- 
formed better in a difficult spatial mapping task when they 
had previously heard spatial terms describing the arrays. 
Hermer- Vazquez and her colleagues (Hermer- Vazquez, 
Moffet, & Munkholm, 2001; Hermer-Vazquez, Spelke, & 
Katsnelson, 1999) found that performance on a difficult 
spatial retrieval task was correlated with the ability to use 
the spatial language relevant to the task (but see Cheng & 
Newcombe, 2005, and Learmonth, Nadel, & Newcombe, 
2002). Such findings suggest a role for spatial relational 
language in supporting spatial cognition. 

In the present research, we asked whether the pres- 
ence of spatial language—specifically, English spatial 
prepositions—can influence the encoding and memory of 
pictorial scenes. In answering this question, we sought ev- 
idence bearing on two questions: (1) whether participants’ 
memories showed language-related alterations (Experi- 
ments | and 2), and (2) whether these alterations resulted 
from an active comparison of language and the picture in 
pursuit of a common construal of the two, a process we 
refer to as interactive encoding (Experiment 3). 

We chose spatial prepositions for several reasons. First, 
spatial prepositions themselves have been the focus of 
much recent research (e.g., Bowerman & Pederson, 1992, 
1996; Coventry, Carmichael, & Garrod, 1994; Coventry 
& Garrod, 2004; Feist, 2000; Levinson, Meira, & the 
Language and Cognition Group, 2003; Pederson et al., 
1998; Regier, 1996; Vandeloise, 1991). Second, spatial 
prepositions are used with very high frequency even in 
nonspatial domains because space is a base domain for 
many metaphors (e.g., Boroditsky, 2001; Gentner & Imai, 


1992; Gentner, Imai, & Boroditsky, 2002). Third, there 
has been comparatively little work on the possible effects 
of prepositions on the encoding of static spatial relations 
to date. Yet, spatial prepositions exhibit considerable vari- 
ability across languages (Bowerman & Pederson, 1992, 
1996; Feist, 2000; Levinson et al., 2003). Coupled with 
evidence of very early learning (Bowerman & Choi, 2003; 
Choi & Bowerman, 1991), this variability raises the pos- 
sibility that the semantic encodings in spatial prepositions 
might be especially likely to influence speakers’ construals 
of the world. 


Design of the Experiments 

The basic plan was as follows. For each of the preposi- 
tions tested, we created a sentence (e.g., “The block is on 
the building”’) and a triad of pictures that varied in how 
well they fit the sentence (see Figure 1). The standard pic- 
tures were designed to be possible borderline exemplars 
of the spatial term, but not to be central exemplars. (In 
Norming Study 2B, we used spontaneous verbal descrip- 
tions to confirm that the standards did not by themselves 
give rise to the key prepositions.) For each standard, there 
were two variants, one of which (the p/us variant) was a 
better exemplar of the spatial term, and one of which (the 
minus variant) was a poorer exemplar. Thus, the standard 
was designed so that the spatial preposition could apply 
to it, and the two variants were either more typical of the 
core prepositional category or less so. The participants 
were shown the set of standards and later were asked to 
recognize which picture they saw. 

If participants apply the semantic categories of the 
spatial terms when encoding the standard, they should be 
more likely to have false alarms (FAs) to the plus vari- 
ant than to the minus variant; that is, they should show a 
shift in recognition toward the category’s center. Across 
three experiments, we varied the input conditions and 
asked under what conditions participants would show this 
effect. If we were to see language effects only when the 
participants were provided with language at encoding, 
this would provide support for a thinking-for-language 
hypothesis—a generalization of Slobin’s thinking-for- 
speaking hypothesis to encompass comprehending as well 
as producing language. If, on the other hand, we were to 
see spatial category effects even without the presentation 
of language at encoding, this would support the possibility 
that language influences cognition in a more far-reaching 
manner. 

The goal of the first two experiments was to test whether 
the presence of spatial language during encoding influences 
picture recognition. The aim of Experiment 3 was to extend 
the scope of the findings, as well as to clarify the explana- 
tion. We tested whether picture encoding can be influenced 
not just according to the specific terms but according to the 
dimensions on which the terms operate. In other words, in- 
stead of asking whether the use of on will shift the encoding 
of an ambiguous picture toward the center of the category 
named by on, in Experiment 3, we asked whether the use of 
any spatial term would highlight the spatial semantic sys- 
tem, shifting the encoding of all ambiguous pictures toward 
the centers of the nearest spatial semantic categories. The 
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Plus variant 


Standard 


Minus variant 


The block is on the building. 


b 
TW 


The balloon is on the table. 


Figure 1. Example picture triads together with the sentence used for each 
triad. In each case, the plus variant was closer than the standard to the center 
of the spatial category conveyed by the sentence and the minus variant was 


farther. 


study also allowed us to test whether our findings are ex- 
plainable by intrusions from a separate encoding of the lan- 
guage used in the task, instead of by an interaction between 
language and the pictures (as we propose). 


EXPERIMENT 1 


In Experiment 1, we addressed the question of whether 
the presence of spatial language at encoding would influ- 
ence recognition memory for simple spatial scenes. The 
participants viewed pictures depicting static spatial rela- 
tions (e.g., a marionette standing on a table, or a coin in 
the palm of a hand). After participating in a 10-min dis- 
tractor task, the participants performed a recognition task 
that included the original pictures and two variants. 

In Experiment 1A, half the participants received two 
sentences along with each picture during the study por- 
tion and chose which best described the picture; the other 
half of the participants received only the pictures. One 
sentence (the target sentence) was plausible, and the other 
(the distractor sentence) was designed to be obviously 
wrong; its purpose was simply to cause the participants 
to read the correct sentence and encode the target spa- 
tial relational term. To forestall inducing a strategy of fo- 
cusing directly on the preposition, only the nouns were 
changed in the distractor sentences. Thus, for the target 
sentence “The puppet is on the table,” the distractor sen- 
tence was “The computer is on the desk.” In construct- 
ing the materials, we ensured that the distractor sentence 
involved the same sense of the preposition as the target 
sentence (e.g., on to describe support from below and not 
support via adhesion). We also ensured that the standards 
did not by themselves give rise to the key spatial seman- 
tic category (see Norming Study 2B). The participants in 
Experiment 1B were instructed to attend carefully to the 


pictures, whereas the participants in Experiment 1C were 
presented with the pictures along with the sentences from 
Experiment 1A, with the prepositions removed. 

The recognition test included all three pictures—the 
standard, the plus variant, and the minus variant—presented 
individually in random order. To the extent that language 
influences recognition memory, we expected to see more 
plus FAs than minus FAs. 


Experiment 1A 


In Experiment 1A, half the participants (the spatial sen- 
tences group) received the sentences along with the pic- 
tures; the other half (the control group) received only the 
pictures. Both groups were told to remember the pictures 
for a later test. If strong linguistic determinism is correct— 
that is, if the semantic categories of one’s language influ- 
ence cognitive encoding even when language is not being 
used—we should see more plus FAs than minus FAs for 
both groups. If the thinking-for-language hypothesis is 
correct, we should see a plus advantage for the spatial sen- 
tences group only. Finally, if there is no effect of language 
on recognition memory, there should be no difference in 
the rates of plus and minus FAs for either group. 


Method 

Design. Encoding condition (spatial sentences/control), a between- 
participants factor, was crossed withrecognition item type (plus variant/ 
standard/minus variant), a within-participants factor. 

Participants. Thirty-six Northwestern University undergraduates 
received course credit for their participation in this experiment. 

Materials. There were 13 triads of pictures and corresponding 
sets of sentences as well as 18 filler pictures and sentences. Each triad 
contained one picture whose figure—ground relation was ambiguous 
with respect to the preposition in the sentence (the standard), one that 
fell closer to the center of the spatial category named by the preposi- 
tion (the plus variant), and one that fell outside of the spatial category 
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named by the preposition (the minus variant) (see Figure 1).! All three 
pictures involved the same objects; the only source of variation was 
the spatial relation between the two objects. In preparing the pictures, 
every attempt was made to guard against a possible recognition bias 
for the plus variant (see the norming studies for Experiment 2). The 
standard from each triad was used for the study portion of the experi- 
ment; all three pictures in the triad were used for the recognition test. 
For each standard, there was a pair of sentences. One described the 
figure-ground relation using the key preposition, and the other used 
the same preposition in the same sense but with two clearly incorrect 
nouns. The target and distractor sentences are presented in Table 1. 

Procedure. The procedure comprised two parts: (1) study and 
(2) recognition. 

Part 1: Study. Twenty-five pictures (13 standards and 12 fillers) 
were randomized and presented individually for 5 sec each on a 
computer screen. All participants were told that they would be tested 
later on their ability to recognize these pictures. 

To ensure that the participants in the spatial sentences group pro- 
cessed the sentences, we asked them to choose which of two sentences 
best described the picture. They were provided with answer sheets 
with two sentences for each picture: the target sentence and a distrac- 
tor sentence, as described above. The participants in the control condi- 
tion were given no additional instructions. After completing the study 
portion, the participants did an unrelated filler task for 10 min. 

Part 2: Recognition. All participants received the same yes/no rec- 
ognition test. All three of the pictures in each triad were presented in- 
dividually in random order on a computer screen along with 12 fill- 
ers (6 old and 6 new), for a total of 51 pictures. The participants were 
asked to indicate on a numbered answer sheet whether they had seen 
each picture during the earlier study portion. Each picture remained 
on the screen until the participant pressed a key to continue. 


Results 

The two groups did not differ significantly in their over- 
all error rates, nor in the overall rates of the two kinds of 
possible errors, misses and FAs (see Figure 2). 

The key prediction was that the pattern of FAs would 
be influenced by the presence of spatial language at study. 
Indeed, the participants in the spatial sentences condition 
were significantly more likely to have FAs to the plus 
variant than to the minus variant (see Figure 3) [#(17) = 
5.32, p < .0001]. The participants in the control condi- 
tion showed no such difference in their FA rate [¢(17) = 
—0.72, p > .10]. Thus, having spatial language present 
at encoding led to a skewing of recognition errors toward 
the core of the spatial category, providing evidence for an 
effect of spatial language on the encoding of the pictures. 


Discriminability analysis. To further test this claim, 
we calculated two d’ measures for each participant: one for 
the discriminability of the minus variant from the standard 
and one for the discriminability of the plus variant from the 
standard. The larger of the two was then determined,” and 
the participants were pooled by condition. Table 2 shows 
the results (along with those for Experiments 1B and 1C). 
The control condition showed a symmetric discriminabil- 
ity pattern, with equal numbers of participants showing 
the two possible discriminability biases. In contrast, the 
spatial sentences condition showed a strongly skewed 
discriminability pattern: 12 participants showed greater 
discriminability for the minus variant, and none showed 
greater discriminability for the plus variant, a significant 
difference between conditions [y2(2, N = 35) = 9.65, p < 
.01]. As a further check on the effect of condition on dis- 
criminability, the set of d’ data was subjected to a 2 X 2 
multivariate ANOVA. In support of our conclusion, we 
found an effect of condition [F(1,34) = 10.72, p = .002]. 


Discussion 

When spatial language was present at encoding, mem- 
ory for the spatial relations in our pictures was systemati- 
cally shifted in the direction of the spatial preposition. This 
is evidence for at least the moderate thinking-for-language 
version of the Whorfian hypothesis. We argue that this 
shift reflects an interactive encoding process in which the 
representation of the pictures is influenced by the spatial 
sentences. In the next experiment, we asked whether peo- 
ple would spontaneously invoke spatial language when 
faced with a difficult encoding task. Extrapolating from 
Brown and Lenneberg’s (1954) finding that the codability 
of colors influenced people’s ability to remember them 
later, we reasoned that if participants believe they are pre- 
paring for a difficult memory task, they might mentally 
invoke linguistic encodings as a memory aid. 


Experiment 1B 


In Experiment 1B, we asked whether there are language 
effects on recognition memory without the overt use of 
language at encoding. We tested the possibility that par- 
ticipants instructed to pay careful attention to the pictures 
at study might be induced to encode the pictures linguisti- 


Table 1 
Sentences Used in Experiments 1 and 2 


Target Sentences 


Distractor Sentences 


The firefly is on the wagon wheel. 
The puppet is on the table. 

The chair is in the corner. 

The dirt is on the dump truck. 
The ball is under the chair. 

The block is on the building. 

The balloon is on the stick. 

The firefly is in the dish.* 

The coin is in the hand.* 

The spider is in the bowl of apples. 
The plane is on the ground. 

The hose is around the tree trunk. 
The balloon is on the table. 


The watch is on the nightstand. 
The computer is on the desk. 

he house is in the middle. 

he snow is on the mountain. 
he table is under the lamp. 

he plant is on the shelf. 

he kite is on the string. 

he house is in the valley. 

he spaceship is in the crater. 
he shirt is in the basket of laundry. 
he woman is on the floor. 

he armband is around the arm. 
The antenna is on the roof. 


Ss3s3353555 


“Sentences altered for Experiment 2. 
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Figure 2. Error rates in Experiments 1A, 1B, and 1C. 


cally. If so, they should be more likely to have FAs to the 
plus variant than to the minus variant, as did the spatial 
language participants in Experiment 1A. 


Method 

Participants. Eighteen Northwestern University undergraduates 
received course credit for their participation in this experiment. 

Materials. The materials were those used in Experiment 1A. 

Procedure. The procedure was as in the control condition in 
Experiment 1A, except that the participants were instructed to pay 
careful attention to the pictures because the recognition test would 
be very difficult. After completing the study portion, the participants 
did an unrelated filler task for 10 min. They were then given the 
same recognition test as in Experiment 1A. 


Results and Discussion 

The overall error rate observed in Experiment 1B (VM = 
.28, SD = .09) was lower than that observed in Experi- 
ment 1A (VM = .34, SD = .08), suggesting that the par- 
ticipants did pay more careful attention to the pictures 
during study (see Figure 2). In particular, the participants 
in Experiment 1B had a lower incidence of FAs than did 
the participants in either of the conditions from Exper- 
iment 1A [Experiment 1A control, M = .41, SD = .10; 


Experiment 1A spatial sentences, M = .38, SD = .08; 
Experiment 1B, M = .29, SD = .08; comparison with 
control, (34) = 3.78, p < .001; comparison with spatial 
sentences, ¢(34) = 2.91, p < .01]. 

However, we found no evidence that the participants 
in Experiment 1B mentally invoked spatial language as a 
memory aid. They did not show the error pattern character- 
istic of spatial language use (more plus than minus FAs). 
Instead, like the control group in Experiment 1A, the par- 
ticipants in Experiment 1B showed roughly equal num- 
bers of plus and minus FAs [¢(17) = —1.27, p = .22]. 

So far, we have evidence for the influence of spatial 
language when it is explicitly presented, although not for 
the stronger possibility that language can affect encoding 
and recognition when it is not overtly present. In Experi- 
ment IC, we tested the specificity of the language effect. 
If the recognition shift is due to spatial language, then we 
should not see this shift if participants are given verbal 
descriptions that do not contain spatial language. 


Experiment 1C 


Experiments | A and 1B provided evidence for a thinking- 
for-language version of the language-and-thought hypoth- 


Rate of False Alarms 
wo 


—¢e— Spatial sentences 
—®— Control 

—4— Attention 

—<<— Object sentences 


Plus 


T 
Minus 


Picture Variant 


Figure 3. False alarms by condition in Experiments 1A, 1B, and 1C. 
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Table 2 
Participants Pooled According to the Discriminability Analysis 
in Experiments 1A, 1B, and 1C 


d’ Larger d’ Larger d 

Group for Plus for Minus Equal 
Control 4 4 10 
Spatial sentences 0 12 5 
Attention* 8 4 5 
Object sentences* 6 3 7 


“One participant was dropped from the analysis due toahitrateof 1. *One 
participant was dropped from the analysis due to a hit rate of 1, and another 
was dropped due to a plus false alarm rate of 0. 


esis: that using spatial language during the encoding of 
scenes fosters spatial representations congruent with the 
language. In Experiment 1C, we investigated the reverse 
possibility: that using language that emphasizes only the 
objects present (i.e., eliminating the spatial language) will 
invite encodings that are primarily centered on the objects. 
Therefore, participants given object-centered language will 
experience no shift toward the core of the spatial category 
(and thus will not show any differential tendency to have 
FAs to the plus variants over the minus variants). Further- 
more, if language about the objects induces object-centered 
encodings, then we might find an overall loss of accuracy in 
the ability to retain the spatial relations, leading to depressed 
performance on the recognition task (in which participants 
must reject alternatives that share objects but differ slightly 
in their spatial relations). Although this outcome seems 
most plausible, there is another possibility: It could be that 
the presence of any linguistic stimuli at encoding will en- 
train participants into a linguistic mode and prompt them 
to think of the semantic category that best fits the standard. 
If the presence of language induces a general mind-set to 
invoke applicable semantic encodings, participants given 
language at encoding (even nonspatial language) should 
show a greater rate of plus FAs than minus FAs. 

We tested these hypotheses by presenting the partici- 
pants with sentences during encoding that simply pointed 
out the objects without spatial prepositions. Equal plus 
and minus FAs in this case would support a semantically 
specific version of the thinking-for-language hypothesis, 
whereas a prevalence of plus FAs would support a more 
general effect of language. 


Method 

Participants. Eighteen Northwestern University undergraduates 
received course credit for their participation in this experiment. 

Materials. The pictures were the same as those in Experiment 1A. 
We modified the sentences on the participants’ answer sheets from 
those used in Experiment 1A by removing the prepositions (e.g., 
“The picture shows a block and a building” and “The picture shows 
a plant and a shelf”’). 

Procedure. The procedure was identical to that in the spatial sen- 
tences condition in Experiment 1A: The participants saw pictures 
and circled the appropriate sentence; they then did a 10-min filler 
task and, finally, were given a recognition test. 


Results and Discussion 
The participants who were given object sentences at en- 
coding made more errors than did those told to pay close 


attention (Experiment 1B) or those given spatial sentences 
(Experiment 1A); in particular, they were more likely to have 
FAs than were the participants in the attention condition (see 
Figure 2). This recognition disadvantage was confirmed by 
independent samples ¢ tests [total errors for spatial vs. object 
sentences, ¢(34) = —2.04, p < .05; total errors for attention 
vs. object sentences, 7(34) = —3.35, p < .005; FAs for at- 
tention vs. object sentences, t(34) = —3.82, p < .001]. 

However, object sentence participants failed to show 
any recognition shift toward the core of the spatial cat- 
egory designated by the preposition. They demonstrated 
equal plus and minus FAs [M = .43 (SD = .18) and M = 
47 (SD = .15), respectively; (117) = —0.94, p = .36], 
much as did the no-language participants in the previous 
experiments. This pattern differed from the pattern shown 
by spatial sentences participants in Experiment 1A. This 
provides support for the suggestion that the spatial prep- 
osition was specifically responsible for the pattern of 
responses observed in the language condition in Exper- 
iment 1A, a thinking-for-language result. The complete 
FA breakdown for Experiment | is presented in Figure 3. 

Discriminability analysis. In order to compare across 
the four conditions, two d’ measures were calculated for 
each individual participant in Experiments 1B and 1C 
(as in Experiment 1A): one for the discriminability of 
the minus variant from the standard, and one for that of 
the plus variant from the standard. The larger of the two 
was then determined, and the participants were pooled 
by condition (see Table 2). For both the attention group 
(Experiment 1B) and the object sentences group (Experi- 
ment 1C), we found that the participants more often failed 
to discriminate the minus variant (which was designed to 
be perceptually more similar to the standard) than the plus 
variant; that is, the minus variant was more confusable 
with the standard than was the plus variant. This contrasts 
with the pattern found for the spatial sentences group 
(Experiment 1A), for which the plus variant was more 
confusable with the standard than was the minus variant. 
This difference in discriminability between conditions 
was significant [y2(6, N = 68) = 19.31, p < .01]. 

As another comparison across the four conditions, we 
conducted a 4 (condition) X 2 (d’ type: plus or minus) 
ANOVA over the d’ data. As predicted, there was an interac- 
tion between condition and d’ type [F(3,64) = 5.67, p = 
.002]. Post hoc ¢ tests revealed that the standard was more 
confusable with the plus variant than with the minus variant 
for the spatial sentences group [t(16) = —5.52, p < .0001 by 
a Bonferroni correction]; the other three conditions showed 
no difference. The interaction of condition by d’ type did 
not reach significance in the items analysis [F(3,36) = 2.34, 
p = .09]. However, t tests showed the same pattern as for the 
participants analysis: the standard was more confusable with 
the plus variant than with the minus variant for the spatial 
sentences group only [¢(7) = —4.81, p = .002]. 


EXPERIMENT 2 


In Experiment 1, we found that recognition memory for 
spatial scenes was influenced by spatial language presented 
at study. These results provide support for the interactive 
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encoding claim that the participants adjusted their encoding 
of the picture to better accord with the spatial preposition. 

However, in Experiment 1, the participants received all 
three recognition items (standard, plus, and minus) for each 
triad (randomly ordered) in the yes/no recognition task. This 
leaves open the possibility that responses to a given item 
might have been influenced by having seen other items from 
the same triad during the recognition task. In Experiment 2, 
we changed the design of the recognition test to eliminate this 
possibility. We replicated the basic plan of Experiment 1A, 
but, in the recognition test, we presented each participant 
with only one recognition item from each triad. In addition, 
to ensure that the materials met the requirement of the de- 
sign, we performed three norming studies. 


Method 

Design. The factors were encoding condition (spatial sentences/ 
control; between participants), recognition item type (plus variant/ 
standard/minus variant; within participants), and assignment condi- 
tion (between participants). 

Participants. One hundred eighteen Northwestern University 
undergraduates received course credit for their participation. An ad- 
ditional 86 Northwestern University undergraduates participated in 
the norming studies. 

Materials. We preformed three norming studies on the mate- 
rials used in Experiment 1. In Norming Study 2A, we asked 24 
Northwestern University undergraduates to rate the applicability 
of the spatial sentences to each of the items in order to verify that 
the spatial sentences were most applicable to the plus variants and 
least applicable to the minus variants, with intermediate applica- 
bility to the standard pictures. We found that, as expected, partici- 
pants gave the highest ratings to the plus variants (IM = 5.72, SD = 
.70), followed by the standards (M = 4.47, SD = 1.16), and then by 
the minus variants (VM = 2.54, SD = 1.63) [F(2,36) = 22.35, p< 
.0001]. However, examination of the results for individual triads 
showed a different pattern for two of the triads: one depicting a coin 
ina hand, and one depicting a firefly in a dish. These sentences were 
adjusted accordingly for Experiment 2. The changed sentences are 
indicated with asterisks in Table 1. 

In Norming Study 2B, we collected free descriptions of the stan- 
dards (from 12 Northwestern University undergraduates) and plus 
variants (from 10 Northwestern University undergraduates) in order 
to check whether (1) the plus variants would elicit the target spatial 
prepositions from Experiment 1A and (2) the standards would be 
less likely to elicit consistent spatial prepositions, indicating that they 
were indeed ambiguous with respect to the spatial sentences. As ex- 
pected, the spontaneous descriptions varied considerably, particularly 
for the standards. The rate of use of the key spatial prepositions was 
.25 in descriptions of the standards (SD = .25). Also as expected, 
there was a trend for the key prepositions to be used more often in 
descriptions of the plus variants (M = .32, SD = .32), although this 
difference did not reach significance [t(12) = —1.01, p > .10]. 

In Norming Study 2C, we collected ratings of the degree of simi- 
larity between the plus variant and the standard and between the 
minus variant and the standard from 40 Northwestern University 
undergraduates to verify that both variants were equally similar to 
the standard. We were especially concerned to rule out an imbalance 
in similarity in favor of the plus variant, since this could provide an 
alternate explanation for the plus FAs. Although the overall ratings 
were comparable for the two alternatives, participants gave slightly 
higher similarity ratings to the minus variants (M = 4.96, SD = 
1.40) than to the plus variants (VM = 4.73, SD = 1.51) [¢(39) = 
—0.47, p < .0001]. Across individual triads, the minus variant was 
judged more similar to the standard for five triads, the plus vari- 
ant was judged more similar for three triads, and they were judged 
equally similar for five triads. 


As a result of the norming studies, we made minor modifications 
to two of the triads of pictures for use in Experiment 2,3 and a change 
of preposition (from in to on) in the sentences corresponding to two 
others (indicated with asterisks in Table 1). One triad (depicting a 
balloon ona stick) was eliminated so that the number of triads would 
be divisible by three as required by the design. 

Procedure. The study procedure was identical to that of Experi- 
ment 1A. Also as before, both conditions received the same yes/no rec- 
ognition test after a 10-min filled delay. However, each participant saw 
only one picture from each triad at recognition (instead of all three as 
in Experiment 1). These were presented in random order along with 12 
fillers (6 old and 6 new) on a computer screen. As in Experiment 1, the 
participants were asked to indicate on a printed answer sheet whether 
they had seen each picture during the earlier study portion. Each pic- 
ture remained on the screen until the participant pressed a key. 


Results 

As in Experiment 1A, we found the predicted effect of 
spatial language. The participants in the spatial sentences 
condition were significantly more likely to have FAs to 
the plus variant than to the minus variant, whereas partici- 
pants in the control condition showed no such asymmetry 
(Figure 4). The difference between the plus FAs and the 
minus FAs was significant only in the spatial sentences 
condition [¢(57) = 6.87, p < .0001]. In addition, the dif- 
ference in the rate of FAs between the two groups only 
reached significance for the responses to the plus variant 
[t(116) = —3.93, p = .0001]. 

Also as in Experiment 1A, we found no difference in the 
miss rates between the control and spatial sentences groups 
[t(116) = 1.46, p > .10]. In contrast to Experiment 1A, 
the overall FA rates for the two groups in Experiment 2 did 
differ (Figure 5), with FAs being more prevalent among the 
participants in the spatial sentences condition [¢(116) = 
—1.77, p < .05, one-tailed]. As the analysis of the pattern 
of FAs (above) shows, this difference was entirely attribut- 
able to a difference in responses to the plus variants. 

Discriminability analysis. As in Experiment 1, two 
d’ measures were calculated for each individual partici- 
pant—one for minus and one for plus. The larger of the 
two was then determined, and the participants were pooled 
by condition (see Table 3). 

The results of the d’ analysis for Experiment 2 replicate 
those for Experiment 1. In the spatial sentences condition, 
the dominant pattern was one of greater discriminability 
between the standard and the minus variant than between 
the standard and the plus variant. However, in the control 
condition, an equal number of participants had a larger 
minus d’ as had a larger plus d’, a significant difference 
between conditions [2(2, N = 118) = 16.67, p < .0001]. 
As a further check on the effect of condition on discrim- 
inability, the set of d’ data was subjected to a 2 X 2 multi- 
variate ANOVA. In support of our conclusion, we found an 
effect of condition [F(2,115) = 6.63, p = .002]. 


EXPERIMENT 3 


Our experiments so far have provided evidence for the 
hypothesis that language interacts with visual input to 
influence the encoding of pictures. In Experiment 2, as 
in Experiment 1, the recognition results showed a shift 
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Figure 4. False alarms by condition in Experiment 2. 


toward the core of the spatial category when spatial lan- 
guage was present at encoding, consistent with the inter- 
active encoding account. 

However, before drawing strong conclusions, we need 
to address a possible alternate explanation for these ef- 
fects, which we term the separate encoding account. Be- 
cause the effect of language in Experiments | and 2 was 
to shift recognition in the direction of the specific terms 
present at encoding, it could be that the effect was simply 
due to memory for the language presented, rather than 
to differential encoding of the pictures. This line of rea- 
soning has been articulated most clearly in research on 
memory overlay effects, including effects of verbal ques- 
tioning on eyewitness testimony (Loftus & Palmer, 1974; 
McCloskey & Zaragoza, 1985) and effects of language 
(Carmichael, Hogan, & Walter, 1932) and of verbal over- 
shadowing (Schooler & Engstler-Schooler, 1990) on the 
reproduction of visual materials. In this arena, the issue of 
whether language alters prior visual memories or merely 
exists as an alternate, possibly more available encoding 
has been the source of much debate. In Loftus and Palm- 
er’s classic study, participants viewed a filmed automo- 
bile accident, then answered questions about what they 
had viewed. The results showed that the phrasing of the 


questions (e.g., use of the verb smash rather than the verb 
collide) influenced the likelihood that participants would 
falsely claim to have seen a given element (e.g., broken 
glass), suggesting that the subsequent language had al- 
tered participants’ memorial representation of the event. 
However, McCloskey and Zaragoza argued that the results 
could have come about without a direct effect of verbal in- 
formation on pictorial memory. To demonstrate this, they 
presented participants with a scene that included Target 
Object A and then referred to it as Object B in postevent 
questioning. At test, participants were asked to determine 
whether they had seen Object A, which was present in the 
scene, or Object C, which was neither present in the scene 
nor mentioned in postevent discussion. They found that 
misleading information did not result in lower accuracy 
in this task (in contrast to the case when A was contrasted 
with B at test), from which they concluded that the verbal 
information had not changed participants’ memory for 
the original event. As this research shows, finding effects 
of language on retrieval of visually presented informa- 
tion does not necessarily imply effects of language on the 
visual representation itself. Such effects could also arise 
from two separately encoded and stored memory traces, if 
participants draw on their verbal memory to supplement 
their visual memory. On this account, the results of Ex- 
periments | and 2 (1.e., the elevation in plus FAs) would 
be explained by people resorting to their stored language 
trace when their memory for a picture is weak, rather than 
by an interactive encoding process by which language in- 
fluences the encoding of the pictures. 

One important difference from the present research is that 
McCloskey and Zaragoza’s (1985) discussion was directed 
at studies in which the visual materials were encoded first, 
with verbal descriptions following later. Thus, they asked 
whether later verbal input can alter a prior visual memory, 
whereas, in our experiments, we tested for interactions of 
verbal and visual materials presented together. In other 
words, we are arguing for an interactive encoding process, 
as opposed to the retroactive alteration process that was the 
focus of McCloskey and Zaragoza’s critique. But obviously, 
the separate encoding explanation could still apply. 

Experiment 3 was designed with two goals in mind: to 
test whether our language effect could be accounted for 
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Figure 5. Error rates in Experiment 2. 
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Table 3 
Participants Pooled According to the 
Discriminability Analysis in Experiment 2 


d’ Larger d’ Larger d’ 

Group for Plus for Minus Equal 
Control 20 20 20 
Spatial sentences 4 38 16 


by the separate encoding possibility and (if possible) to 
extend the scope of the language effect. Happily, there is 
a way to accomplish both these goals at once. Rather than 
testing for narrow, direct effects of language (i.e., whether 
participants’ picture encodings shift in the direction of the 
specific terms used), we tested for a more global effect, 
using a task inspired by McCloskey and Zaragoza’s (1985) 
technique. We asked whether participants will show indi- 
rect effects of using nonappropriate spatial terms. Spe- 
cifically, in Experiment 3, we addressed the question of 
whether introducing a nonappropriate spatial term will 
call attention to spatial semantics more generally and lead 
to a shift toward the more appropriate spatial category. 


Experiment 3A 


In Experiment 3A, the participants saw the stan- 
dard pictures immediately preceded by sentences. Half 
the sentences were the key preposition sentences of the 
earlier experiments. The other half were clearly not ap- 
propriate—that is, they expressed a very different spatial 
relation from the one that best fit the scene. For example, 
for the standard in Figure 1, the key preposition sentence 
(the plausible description) was “The block is on the build- 
ing” and the nonappropriate preposition sentence was 
“The block is in the building.” The participants were told 
to read the sentences and to remember the pictures for 
a later recognition task. (Obviously, given this instruc- 
tion, the participants could have ignored the sentences; in 
Experiment 3B, we verified that they did not.) They were 
then given a two-alternative forced choice (2AFC) recog- 
nition test between the actually presented standard and the 
plus variant (incorrect). On the basis of our prior results, 
for pictures that were coupled with the key preposition, 
we expected the participants to (incorrectly) choose the 
plus variant. The question was what will happen with the 
nonappropriate prepositions? 

Suppose that the interactive encoding account is cor- 
rect and, further, that hearing a spatial term calls forth not 
just the specific category associated with the term but, 
more generally, the semantic set or dimension it belongs 
to. Then, hearing a clearly nonappropriate spatial preposi- 
tion may prompt participants to consider which spatial 
preposition should apply. That is, participants may think 
(for example) “This isn’t in, it’s on.” They will substitute 
the correct (or most applicable) spatial relation—which, 
by design, is the key prepositional category. If this rea- 
soning holds, the effect of being given a nonappropriate 
preposition should be (paradoxically) to increase FAs to 
the plus variants. 

In contrast, the separate encoding account predicts the 
pattern found by McCloskey and Zaragoza (1985): no rise 


in FAs for pictures paired with the nonappropriate prepo- 
sition. The nonappropriate-preposition sentences do not 
match either of the recognition pictures. Thus, if partici- 
pants encode the sentences and pictures separately and, 
during test, resort to the sentences when they forget the 
pictures (as in the separate encoding account), then for 
the nonappropriate preposition sentences, their sentence 
memory will be irrelevant to the choice between the stan- 
dard and the plus variant. They should therefore show no 
elevation in FAs. 

Thus, both accounts predict that participants will often 
have FAs to the plus variant when the picture has been 
coupled with the key preposition; but they make different 
predictions for pictures coupled with the nonappropriate 
prepositions. The interactive encoding account predicts 
a high FA rate for pictures presented with either the key 
prepositions or the nonappropriate prepositions. The sepa- 
rate encoding account predicts a high FA rate only for pic- 
tures presented with the key prepositions. 


Method 


Participants. Twenty Northwestern University undergraduates 
received course credit for their participation. 

Design and Materials. The factors were encoding condition 
(key preposition/nonappropriate preposition, within participants) and 
assignment condition (between participants). Each participant saw 
12 standards and 8 fillers—a total of 20 trials, of which half con- 
tained key-preposition sentences and half contained nonappropriate- 
preposition sentences. 

The pictures and the key-preposition sentences were as in Ex- 
periment 2. The nonappropriate-preposition sentences were created 
from the key-preposition sentences by replacing the preposition with 
another that was incompatible with both the standard picture and the 
plus variant. In selecting these, we avoided nonappropriate preposi- 
tions that could name the minus variant. For example, for the block 
and the building depicted in Figure 1, in which the plus FA is on and 
the minus FA is off, we used in as the nonappropriate preposition. 
The complete set of sentences is presented in Table 4. 

Procedure. The procedure comprised two parts: (1) study and 
(2) recognition. 

Part 1: Study. The pictures were presented individually for 3 sec 
each on a computer screen in randomized order. Each picture was 
preceded by a sentence, which was displayed for 1 sec. The partici- 
pants were told that they would read sentences followed by corre- 
sponding pictures and that they would later be asked to recognize only 
the pictures. For half of the pictures, the key spatial sentence (from 
Experiment 2) was presented; for the other half, a nonappropriate- 
preposition sentence was displayed. For the filler pictures, we used 
incorrect sentences that had the right preposition but the wrong noun 
(.e., the distractor sentences from Experiment 1A). This was done 
to keep the participants from noticing a pattern that the incorrect 
sentences always erred in their spatial prepositions. 

Part 2: Recognition. After a 10-min filled delay, all participants 
received the same 2AFC recognition test. The participants were 
shown each standard paired with its plus variant and asked to click 
on the picture they had seen during the study portion. The pictures 
remained on the screen until the participant made a choice. 


Results and Discussion 

The key prediction of the separate encoding account 
is that the FA rate will be higher for pictures that were 
coupled with key spatial prepositions than for those cou- 
pled with nonappropriate prepositions. In fact, there was a 
nonsignificant trend in the reverse direction [key preposi- 
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tions, M = .12, SD = .17; nonappropriate prepositions, 
M = .16, SD = .15; t(19) = .79, p > .4]. Thus, the results 
argue against the possibility that the language effects in 
Experiments | and 2 were simply due to use of indepen- 
dently encoded key-preposition sentences and buttress the 
interactive encoding account of the language effect. 

In addition, the results extend the interactive encoding 
account beyond purely local effects of words as category 
attractors. In Experiment 3A, hearing nonappropriate 
spatial prepositions apparently led to an indirect language 
effect wherein the participants shifted to the most appro- 
priate spatial category, thus achieving a common construal 
of sentence and picture. 


Experiment 3B 


Experiment 3B was a manipulation check, conducted 
to verify that the participants did indeed attend to the sen- 
tences. To do this, we gave 20 new participants the same 
materials as in Experiment 3A, but we tested memory for 
sentences instead of memory for the pictures. The study 
phase was exactly as in Experiment 3A, including the in- 
struction to remember only the pictures for a recognition 
test. However, after a 10 min filled interval, we admin- 
istered a surprise sentence recognition test. Because we 
wanted to test for memory of all the sentences presented, 
we used a yes/no recognition test rather than a forced 
choice task. Each participant was given a packet contain- 
ing one sentence per study item, and responded “yes” or 
“no” to the question “Did this sentence appear in part 1?” 
Half the sentences were old (i.e., seen during the study 
phase), and half were new. Within each class, half con- 
tained the key preposition, and half contained the nonap- 
propriate preposition. 


Results 

The results showed high sentence recognition accuracy 
(mean proportion correct = .83, SD = .16), allowing us 
to rule out the possibility that the sentences were ignored. 
Memory for the key-preposition sentences (M = .88, SD = 
.16) was nonsignificantly better than memory for the non- 
appropriate-preposition sentences (M = .78, SD = .13). 
Thus, the failure to find a difference in the FA rates for the 
two encoding conditions in Experiment 3A was not due to 
the participants’ having failed to process the sentences. 


Interestingly, we also found evidence that interactive 
encoding works both ways. The participants were more 
likely to misrecognize the sentence when a nonappropri- 
ate preposition had been seen at study (VM = .28, SD = 
.27) than when a key preposition had been seen (VM = .08, 
SD = .21) [t(19) = 2.26, p < .05], suggesting that the 
sentences were adjusted to fit the pictures as well as the 
pictures to fit the sentences. 


Discussion 

According to interactive encoding, people’s encoding 
of pictures can be influenced by accompanying language. 
This hypothesis received initial support from our finding 
that picture recognition was influenced by the presence of 
spatial language during encoding in Experiments | and 2. 
In Experiment 3A, we found evidence that language effects 
can extend beyond the application of specific categories. 
Our results suggest that the language effect may often be 
to focus attention on a particular dimension, rather than to 
impose the stated value on that dimension. 

In this experiment, we also tested a possible alternate 
explanation for the effect of language—that the pictures 
and sentences were encoded separately and that the effect 
of language on recognition resulted from the participants’ 
consulting their sentence memory when pictorial mem- 
ory was weak. Contrary to the prediction of the separate 
encoding account (but consistent with the interactive en- 
coding account), there was no difference in FA rates as 
a function of sentence type; indeed, there was a nonsig- 
nificant imbalance in the opposite direction. The results of 
Experiment 3B verified that the sentences were processed 
and retained. The results further showed that sentences 
that were inconsistent with the pictures (those with nonap- 
propriate prepositions) were more likely to be altered than 
those that were more consistent with the pictures. Both 
these findings are in line with the interactive encoding 
hypothesis. 

We further examined the data across all three experi- 
ments for item effects. However, we found no systematic 
pattern across experiments as to which items showed the 
greatest tendency for picture memory to be affected by 
verbal labels, nor for this effect to correlate (or anticor- 
relate) with the degree to which sentence memory was 
affected by the spatial scenes. 


Table 4 
Sentences Used in Experiment 3 


Key-Preposition Sentences 


Nonappropriate-Preposition Sentences 


The firefly is on the wagon wheel. 
The puppet is on the table. 

The chair is in the corner. 

The dirt is on the dump truck. 
The ball is under the chair. 

The block is on the building. 

The firefly is on the dish. 

The coin is on the hand. 

The spider is in the bow] of apples. 
The plane is on the ground. 

The hose is around the tree trunk. 
The balloon is on the table. 


The firefly is under the wagon wheel. 
The puppet is under the table. 

The chair is below the corner. 

The dirt is under the dump truck. 
The ball is on the chair. 

The block is in the building. 

The firefly is under the dish. 

The coin is under the hand. 

The spider is under the bowl of apples. 
The plane is beneath the ground. 

The hose is beneath the tree trunk. 
The balloon is below the table. 
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The results help refine and extend the thinking-for- 
language account. The effect of language in our experi- 
ments was to focus attention on the spatial relations and 
to alter both the encoding of the scene and the encoding of 
the sentence in the direction of the most consistent prepo- 
sitional category. When a linguistic assertion disagrees 
with the perceptual facts, the effect can be to move the 
encodings to a different value within the same semantic 
set—one in which the language and the scene can agree. 


GENERAL DISCUSSION 


The results of our experiments provide evidence that 
spatial language can influence the way people encode and 
remember spatial relations in visual scenes. Our standards 
were designed to be unlikely members of the key spatial 
category (indeed, the spatial preposition was rarely used in 
spontaneous descriptions; see Norming Study 2B). How- 
ever, the participants given spatial prepositions during 
encoding of the standards showed a shift in picture recog- 
nition toward the core of the spatial category denoted by 
the preposition (Experiments 1A and 2). That is, they were 
significantly more likely to have FAs to pictures closer to 
the center of the prepositional category than to those less 
central. The participants who received the pictures with- 
out spatial sentences at study showed no such shift, even 
when they were asked to pay close attention to the pic- 
tures; their FA rates were symmetrical (Experiments 1B 
and 2). Finally, in Experiments 3A and 3B, we ruled out 
the possibility that these language effects simply reflect 
the use of a separately encoded verbal memory, rather than 
an interactive encoding of sentence and picture. Contrary 
to the separate encoding prediction, the misrecognition 
effect was not restricted to the category directly named in 
the sentence. Rather, we found that hearing even a nonap- 
propriate spatial preposition induced a shift toward the 
key-preposition category. This finding is incompatible 
with the use of an independently encoded verbal store. 
The pattern suggests an interaction between language and 
perception during encoding. 

These experiments link two bodies of work that have 
largely been considered independently. One is the neo- 
Whorfian investigations on effects of language on per- 
ceptual and conceptual representation and reasoning 
reviewed in the introduction. The other is research on 
memory overlay effects. In the latter arena, the issue of 
whether language alters prior visual memories or merely 
exists as an alternate, possibly more available encoding 
has been the source of much debate. The results of our 
experiments show that when spatial language is presented 
simultaneously (or just prior) to visual materials, the inter- 
action between linguistic categories and visual materials 
can influence the encoding of spatial relations in both the 
pictures and the sentences. 

Not all spatial tasks are susceptible to language effects. 
For example, Huttenlocher and her colleagues used a loca- 
tion reproduction task (Crawford, Regier, & Huttenlocher, 
2000; Huttenlocher, Hedges, & Duncan, 1991), in which 
participants saw a dot and a quadrilateral reference ob- 
ject; the dot (but not the quadrilateral) then disappeared, 


and participants indicated where on the screen the dot had 
been. The participants’ errors were skewed toward the 
center of an inferred geometric category, even after rating 
the applicability of a (different) linguistic category to the 
array. One difference between the two studies is that, in 
the Huttenlocher et al. studies, memory was probed im- 
mediately after the disappearance of the dot, whereas we 
imposed a 10-min delay. This delay might have contrib- 
uted to the language effect. In addition, we speculate that 
the language effects may be more likely with naturalistic 
materials such as those used in our task than with simple 
geometric objects such as those used by Huttenlocher and 
her colleagues. 

Our finding of an interaction between verbal and vi- 
sual encodings can also be related to recent accounts of 
language comprehension that propose that sentences are 
understood by constructing perceptual simulations of the 
events described (Barsalou, 1999; Glenberg, 1997; Zwaan, 
2004). Such accounts propose a close interaction between 
perceptual and semantic processing. For example, Zwaan 
and Yaxley (2003) found that spatial iconicity affects se- 
mantic relatedness judgments (e.g., people were faster to 
judge BASEMENT and ATTIC as semantically related words 
when they saw ATTIC above BASEMENT than when they saw 
BASEMENT above ATTIC). Zwaan, Madden, Yaxley, and Ave- 
yard (2004) have also shown effects of language on per- 
ceptual judgments. For example, participants were shown 
pictures of objects and asked to judge whether the second 
object was the same as the first. Before each picture pair, 
participants heard sentences such as “You tossed the beach 
ball over the sand toward the kids” or “The kids tossed 
the beach ball over the sand toward you.” Responses were 
faster when the meaning of the sentence matched the size 
difference of the objects. That is, if the sentence described 
a ball moving toward the viewer, participants were faster 
to respond when the second picture showed a ball that was 
larger than the first. The authors suggest that, in this case, 
the thoughts triggered by the sentence were compatible 
with the size discrepancy between the balls, allowing a 
fast response. They concluded that “words can, indeed, 
move mental representations.” 

According to this view, we activate positional and ori- 
entational information when presented with linguistic 
stimuli (e.g., Zwaan et al., 2004; Zwaan & Yaxley, 2003). 
In the case of spatial prepositions, this information might 
correspond to core members of the spatial prepositional 
categories, leading to the observed shift in picture repre- 
sentation toward the center of the prepositional category 
when a spatial preposition has been processed during 
encoding. 

To return to the neo-Whorfian literature, our findings 
are most compatible with a thinking-for-language hy- 
pothesis (see Slobin, 1996), whereby language influences 
thought when one is either producing or comprehending 
language. Across our studies, we saw no evidence that 
people engaged in the covert use of language to encode 
the pictures, even when they were told to pay close at- 
tention. Of course, our design, with its deliberately am- 
biguous standards, might have underestimated the effects 
of language. The low rate of spontaneous use of the key 
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prepositions in the free description task (Norming Study 
2B) suggests that even if the participants did encode the 
pictures linguistically in the attention task, they were 
unlikely to have used our spatial sentences.4 However, 
overall, we have evidence for language effects only when 
language is present. 


Extended Effects 

We found that processing a clearly nonappropriate 
preposition just prior to a scene led to a kind of contrast 
effect: The participants encoded the scene in terms of a 
competing (and more applicable) spatial prepositional cat- 
egory. This suggests an indirect effect of language, rather 
than a simple direct substitution of the named concept into 
the scene. Here, the use of a linguistic term appears to call 
forth a system of semantic categories, from which partici- 
pants choose the most appropriate member. This effect is 
both more subtle and potentially more pervasive than a 
simple direct insertion effect. 

However, these results invite further questions. Why did 
we find indirect effects of extraneous language when Mc- 
Closkey and Zaragoza (1985) did not? (Recall that McClos- 
key & Zaragoza found that retroactive false information 
concerning a presented object did not impair participants’ 
ability to correctly recognize the object they had viewed, 
as opposed to a new, previously unmentioned, object.) 
One possibility is that language effects—both direct in- 
sertion effects and indirect semantic system effects—are 
more likely when language is presented before or during 
the encoding of the perceptual materials (as in our ex- 
periments) than when the language occurs after the per- 
ceptual materials have been encoded (as in McCloskey & 
Zaragoza’s experiments). This would be consistent with 
the interactive encoding account. 

Another intriguing possibility is that a further contribu- 
tor to the difference between the two studies is the kinds 
of items used. McCloskey and Zaragoza (1985) used ob- 
ject substitution (e.g., they showed a hammer and referred 
verbally to a screwdriver). In contrast, our manipulation 
involved a change of spatial relation. There is reason to 
believe that relational terms such as prepositions (and 
many verbs and adjectives) are more likely to belong to 
systems of interrelated semantic categories than are con- 
crete nouns (Gentner & Boroditsky, 2001; Huttenlocher & 
Lui, 1979). It seems plausible that people are more likely 
to see such interrelated categories as contrastive and to 
shift from one to another. We might therefore expect the 
extended effects explored here to be strongest for rela- 
tional terms and to be much weaker (if they occur at all) 
for object terms. A question for future research is whether 
and how the technique explored here would apply across 
different kinds of semantic systems. 

Finally, we note that the meanings of spatial relational 
terms incorporate nonspatial factors such as support 
and containment relations, intended function, and so on 
(Bowerman & Pederson, 1992; Coventry & Garrod, 2004; 
Coventry, Prat-Sala, & Richards, 2001; Feist, 2000, 2004; 
Feist & Gentner, 2003; Talmy, 1988; Vandeloise, 1991), in 
addition to geometry. Given that spatial meaning is in fact 


quite complex, a question for further research is how these 
factors interact with geometric factors in the encoding of 
spatial scenes. 


Thinking for Language 

This work fits with recent discussions that have made 
finer distinctions within the language-and-thought arena. 
Gentner and Goldin-Meadow (2003) distinguish three 
versions of the claim that language influences thought: 
language as lens, language as tool kit, and language as 
category shaper. The language-as-lens view is the strong 
“Whorfian hypothesis” of linguistic determinism: that 
the language we acquire determines how we perceive and 
represent the world. On the language-as-category-shaper 
view, human categories are relatively universal, but lan- 
guage can influence the boundaries (Papafragou et al., 
2002), implying enduring, if limited, effects on cognition. 
We espouse the language-as-tool-kit view: that acquiring 
a language provides new representational resources that 
augment the capacity for encoding and reasoning (Dennett, 
1993; Gentner, 2003; Vygotsky, 1934/1962). On this view, 
language provides tools that facilitate forming and hold- 
ing particular construals, but it does not replace all other 
encoding formats (Gentner & Goldin-Meadow, 2003). 

Slobin’s (1996) thinking-for-speaking (or more broadly, 
thinking-for-language) hypothesis is essentially orthogo- 
nal to these distinctions, as it concerns when language ef- 
fects occur rather than what the effects are. However, it 
is most compatible with the second and third versions of 
the language-and-thought hypothesis given above. When 
conversing, people are particularly likely (1) to encode 
and reason using the semantic tools provided by their 
language and (2) to honor the semantic distinctions of 
the language. Our findings are compatible with such a 
thinking-for-language account, although of course they 
do not rule out stronger effects of language in other con- 
texts (see Haun, Rapold, Call, Janzen, & Levinson, 2006; 
Majid et al., 2004). Our findings here are less dramatic 
than those predicted by the strong Whorfian hypothesis, 
but they still leave room for pervasive semantic effects, 
given the prominence of language in human mental life. 
As Pinker (1989, p. 360) states, “Whorf was surely wrong 
when he said that one’s language determines how one con- 
ceptualizes reality in general. But he was probably correct 
in a much weaker sense: one’s language does determine 
how one must conceptualize reality when one has to talk 
about it.” 


AUTHOR NOTE 


This work was supported by NSF-LIS Award SBR-97203 13 and NSF- 
ROLE Award 21002/REC-0087516 and was completed while the first 
author was a postdoctoral fellow at Northwestern University. We thank 
Kathleen Braun and Michelle Osmondson for help in conducting and 
analyzing the study, Jonathan Cohen and Benjamin Scott-Hopkins for 
help in preparing the manuscript, and Beth Levin, Jason Jameson, Jeff 
Loewenstein, and Phillip Wolff for helpful discussions of the ideas and 
methods. We also thank Satoru Suzuki for the d’ analysis program, Jeff 
Rouder for discussions of the statistical analyses, and Melissa Bower- 
man for allowing us access to her materials. Finally, we thank Karen 
Emmorey, Nora Newcombe, Rolf Zwaan, and an anonymous reviewer 
for helpful comments on earlier versions of this study. Correspondence 


SPATIAL LANGUAGE INFLUENCES MEMORY = _.295 


concerning this article should be addressed to M. I. Feist, University of 
Louisiana, Institute of Cognitive Science, P.O. Drawer 43772, Lafayette, 
LA 70504-3772 (e-mail: feist@louisiana.edu). 


Note—This article was accepted by the previous editorial team, 
when Colin M. MacLeod was Editor. 


REFERENCES 


BARSALOU, L. W. (1999). Perceptual symbol systems. Behavioral & 
Brain Sciences, 22, 577-660. 

BILLMAN, D., & Krycu, M. (1998). Path and manner verbs in action: 
Effects of “skipping” and “exiting” on event memory. In Proceed- 
ings of the 20th Annual Conference of the Cognitive Science Society 
(pp. 156-161). Mahwah, NJ: Erlbaum. 

BoropiTsky, L. (2001). Does language shape thought? English and Man- 
darin speakers’ conceptions of time. Cognitive Psychology, 43, 1-22. 

BoweERMAN, M., & Cuol, S. (2003). Space under construction: 
Language-specific spatial categorization in first language acquisi- 
tion. In D. Gentner & S. Goldin-Meadow (Eds.), Language in mind: 
Advances in the study of language and cognition (pp. 387-427). Cam- 
bridge, MA: MIT Press. 

BowerMAN, M., & Levinson, S. C. (Eps.) (2001). Language acquisi- 
tion and conceptual development. Cambridge: Cambridge University 
Press. 

BowerRMAN, M., & PEDERSON, E. (1992, December). Crosslinguistic 
perspectives on topological spatial relationships. Paper presented at 
the 91st Annual Meeting of the American Anthropological Associa- 
tion, San Francisco. 

BowerMAN, M., & PEDERSON, E. (1996). Crosslinguistic perspectives 
on topological spatial relationships. Manuscript in preparation. 

Brown, R. W., & LENNEBERG, E. H. (1954). A study in language and 
cognition. Journal of Abnormal & Social Psychology, 49, 454-462. 

CARMICHAEL, L., HoGAN, H. P., & WALTER, A. A. (1932). An experi- 
mental study of the effect of language on the reproduction of visually 
perceived form. Journal of Experimental Psychology, 15, 73-86. 

CHENG, K., & NEWcomBE, N. S. (2005). Is there a geometric module 
for spatial orientation? Squaring theory and evidence. Psychonomic 
Bulletin & Review, 12, 1-23. 

Cuno, S., & BOWERMAN, M. (1991). Learning to express motion events 
in English and Korean: The influence of language-specific lexicaliza- 
tion patterns. Cognition, 41, 83-121. 

CoventrRY, K. R., CARMICHAEL, R., & GARROD, S. C. (1994). Spatial 
prepositions, object-specific function, and task requirements. Journal 
of Semantics, 11, 289-309. 

Coventry, K. R., & GARrop, S. C. (2004). Saying, seeing, and act- 
ing: The psychological semantics of spatial prepositions. New York: 
Psychology Press. 

Coventry, K. R., PRAT-SALA, M., & RICHARDS, L. (2001). The interplay 
between geometry and function in the comprehension of over, under, 
above, and below. Journal of Memory & Language, 44, 376-398. 

CRAWFORD, L. E., REGIER, T., & HUTTENLOCHER, J. (2000). Linguistic 
and non-linguistic spatial categorization. Cognition, 75, 209-235. 

DENNETT, D. C. (1993). Learning and labeling. Mind & Language, 8, 
540-548. 

Feist, M. I. (2000). On in and on: An investigation into the linguistic 
encoding of spatial scenes. Unpublished doctoral dissertation, North- 
western University, Evanston, IL. 

Feist, M. I. (2004, August). Talking about space: A cross-linguistic per- 
spective. Paper presented at the 26th Annual Meeting of the Cognitive 
Science Society, Chicago. 

Feist, M. [I.], & GENTNER, D. (2003, August). Factors involved in the 
use of in and on. Paper presented at the 25th Annual Meeting of the 
Cognitive Science Society, Boston. 

GENNARI, S. P., SLOMAN, S. A., MALT, B. C., & Fircu, W. T. (2002). 
Motion events in language and cognition. Cognition, 83, 49-79. 

GENTNER, D. (2003). Why we’re so smart. In D. Gentner & S. Goldin- 
Meadow (Eds.), Language in mind: Advances in the study of language 
and thought (pp. 195-235). Cambridge, MA: MIT Press. 

GENTNER, D., & Boropitsky, L. (2001). Individuation, relativity, and 
early word learning. In M. Bowerman & S. C. Levinson (Eds.), Lan- 
guage acquisition and conceptual development (pp. 215-256). Cam- 
bridge: Cambridge University Press. 


GENTNER, D., & GOLDIN-MEADow, S. (Eps.) (2003). Language in 
mind: Advances in the study of language and thought. Cambridge, 
MA: MIT Press. 

GENTNER, D., & Imat, M. (1992). Is the future always ahead? Evidence 
for system-mappings in understanding space-time metaphors. In Pro- 
ceedings of the 14th Annual Meeting of the Cognitive Science Society 
(pp. 510-515). Hillsdale, NJ: Erlbaum. 

GENTNER, D., IMat, M., & BoropiTsky, L. (2002). As time goes by: Evi- 
dence for two systems in processing space-time metaphors. Language 
& Cognitive Processes, 17, 537-565. 

GENTNER, D., & Lortus, E. F. (1979). Integration of verbal and visual 
information as evidenced by distortions in picture memory. American 
Journal of Psychology, 92, 363-375. 

GLENBERG, A. M. (1997). What memory is for. Behavioral & Brain Sci- 
ences, 20, 1-55. 

GumPERZ, J. J., & Levinson, S. C. (Eps.) (1996). Rethinking linguistic 
relativity. Cambridge: Cambridge University Press. 

Haun, D. B. M., RAPOLD, C. J., CALL, J., JANZEN, G., & LEVINSON, S. C. 
(2006). Cognitive cladistics and cultural override in hominid spatial 
cognition. Proceedings of the National Academy of Sciences, 103, 
17568-17573. 

HERMER-VAZQUEZ, L., MorFet, A., & MUNKHOLM, P. (2001). Lan- 
guage, space, and the development of cognitive flexibility in humans: 
The case of two spatial memory tasks. Cognition, 79, 263-299. 

HERMER-VAZQUEZ, L., SPELKE, E. S., & KATSNELSON, A. S. (1999). 
Sources of flexibility in human cognition: Dual-task studies of space 
and language. Cognitive Psychology, 39, 3-36. 

Hunt, E., & AGNOLI, F. (1991). The Whorfian hypothesis: A cognitive 
psychology perspective. Psychological Review, 98, 377-389. 

HUTTENLOCHER, J., HEDGES, L. V., & DUNCAN, S. (1991). Categories 
and particulars: Prototype effects in estimating spatial location. Psy- 
chological Review, 98, 352-376. 

HUTTENLOCHER, J., & Lui, F. (1979). The semantic organization of 
some simple nouns and verbs. Journal of Verbal Learning & Verbal 
Behavior, 18, 141-162. 

LEARMONTH, A. E., NADEL, L., & NEWCOMBE, N. S. (2002). Children’s 
use of landmarks: Implications for modularity theory. Psychological 
Science, 13, 337-341. 

Levinson, S. C. (1996). Frames of reference and Molyneux’s question: 
Crosslinguistic evidence. In P. Bloom, M. A. Peterson, L. Nadel, & 
M. F. Garrett (Eds.), Language and space (pp. 109-169). Cambridge, 
MA: MIT Press. 

Levinson, S. [C.] (2003). Language and mind: Let’s get the issues 
straight. In D. Gentner & S. Goldin-Meadow (Eds.), Language in 
mind: Advances in the study of language and cognition (pp. 25-46). 
Cambridge, MA: MIT Press. 

LEVINSON, S. C., Kita, S., HAuN, D. B. M., & Rascu, B. H. (2002). 
Returning the tables: Language affects spatial reasoning. Cognition, 
84, 155-188. 

LEvINSON, S. [C.], MERA, S., & THE LANGUAGE AND COGNITION 
Group (2003). “Natural concepts” in the spatial topological domain— 
Adpositional meanings in crosslinguistic perspective: An exercise in 
semantic typology. Language, 79, 485-516. 

Li, P., & GLEITMAN, L. (2002). Turning the tables: Language and spatial 
reasoning. Cognition, 83, 265-295. 

LOEWENSTEIN, J., & GENTNER, D. (2001). Spatial mapping in preschool- 
ers: Close comparisons facilitate far mappings. Journal of Cognition 
& Development, 2, 189-219. 

LOEWENSTEIN, J., & GENTNER, D. (2005). Relational language and the de- 
velopment of relational mapping. Cognitive Psychology, 50, 315-353. 

Lortus, E. F., & PALMER, J. C. (1974). Reconstruction of automobile de- 
struction: An example of the interaction between language and mem- 
ory. Journal of Verbal Learning & Verbal Behavior, 13, 585-589. 

Lucy, J. A. (1992). Grammatical categories and cognition: A case study 
of the linguistic relativity hypothesis. Cambridge: Cambridge Uni- 
versity Press. 

Magip, A., BOWERMAN, M., Kita, S., HAUN, D. B. M., & LEVINSON, 
S. C. (2004). Can language restructure cognition? The case for space. 
Trends in Cognitive Sciences, 8, 108-114. 

MALT, B. C., SLOMAN, S. A., & GENNARI, S. (2003). Speaking vs. think- 
ing about objects and actions. In D. Gentner & S. Goldin-Meadow 
(Eds.), Language in mind: Advances in the study of language and 
thought (pp. 81-111). Cambridge, MA: MIT Press. 


296 FEIST AND GENTNER 


McC oskey, M., & ZARAGOZA, M. (1985). Misleading postevent in- 
formation and memory for events: Arguments and evidence against 
memory impairment hypotheses. Journal of Experimental Psychol- 
ogy: General, 114, 1-16. 

PAPAFRAGOU, A., MASSEY, C., & GLEITMAN, L. (2002). Shake, rattle, ‘n’ 
roll: The representation of motion in language and cognition. Cogni- 
tion, 84, 189-219. 

PEDERSON, E., DANZIGER, E., WILKINS, D., LEVINSON, S. C., KITA, S., 
& SENFT, G. (1998). Semantic typology and spatial conceptualization. 
Language, 74, 557-589. 

PINKER, S. (1989). Learnability and cognition: The acquisition of argu- 
ment structure. Cambridge, MA: MIT Press. 

REGIER, T. (1996). The human semantic potential: Spatial language and 
constrained connectionism. Cambridge, MA: MIT Press. 

Sapir, E. (1929). The status of linguistics as a science. Language, 5, 
207-214. 

SCHOOLER, J. W., & ENGSTLER-SCHOOLER, T. Y. (1990). Verbal over- 
shadowing of visual memories: Some things are better left unsaid. 
Cognitive Psychology, 22, 36-71. 

SLosin, D. I. (1996). From “thought and language” to “thinking for 
speaking.” In J. J. Gumperz & S. C. Levinson (Eds.), Rethinking 
linguistic relativity (pp. 70-96). Cambridge: Cambridge University 
Press. 

SLOBIN, D. I. (2003). Language and thought online: Cognitive conse- 
quences of linguistic relativity. In D. Gentner & S. Goldin-Meadow 
(Eds.), Language in mind: Advances in the study of language and cog- 
nition (pp. 157-191). Cambridge, MA: MIT Press. 

TaLmy, L. (1988). Force dynamics in language and cognition. Cognitive 
Science, 12, 49-100. 

TayLor, H. A., & TveRsky, B. (1996). Perspective in spatial descrip- 
tions. Journal of Memory & Language, 35, 371-391. 

VANDELOISE, C. (1991). Spatial prepositions: A case study from French 
(A. R. K. Bosch, Trans.). Chicago: University of Chicago Press. 


VyoorTsky, L. S. (1962). Thought and language. Cambridge, MA: MIT 
Press. (Original work published 1934) 

Wuorr, B. L. (1956). Language, thought, and reality: Selected writings 
of Benjamin Lee Whorf (J. B. Carroll, Ed.). Cambridge, MA: MIT 
Press. 

ZWwaaN, R. A. (2004). The immersed experiencer: Toward an embodied 
theory of language comprehension. In B. H. Ross (Ed.), The psychol- 
ogy of learning and motivation (Vol. 44, pp. 35-62). New York: Aca- 
demic Press. 

ZWAAN, R. A., MADDEN, C. J., YAXLEY, R. H., & AVEYARD, M. E. (2004). 
Moving words: Dynamic representations in language comprehension. 
Cognitive Science, 28, 611-619. 

Zwaan, R. A., & YAXLEY, R. H. (2003). Spatial iconicity affects se- 
mantic relatedness judgments. Psychonomic Bulletin & Review, 10, 
954-958. 


NOTES 


1. Three of the triads (those depicting a ball under a chair, a chair in a 
corner, and a hose around a tree trunk) were adapted from drawings in 
Melissa Bowerman’s topological picture series. 

2. All d’ measures within .25 of one another were considered equal. 

3. For the triad depicting a firefly and a wagon wheel, the firefly was 
changed to make its wings visible, making it appear to be flying, the wood 
grain on the wheel was changed to be more realistic, and the background 
color was changed from gray to light blue. For the triad depicting a chair 
and a corner, the lengths of the walls were adjusted to be equal in all three 
pictures, and the distance between the chair and the corner was increased in 
the standard and minus variant, making the pictures more discriminable. 

4. We thank an anonymous reviewer for suggesting this possibility. 
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